Skip to content

Tree cache for timetable loading - #283

Open
Jonas-July wants to merge 86 commits into
motis-project:masterfrom
Jonas-July:timetable-tree-cache
Open

Tree cache for timetable loading#283
Jonas-July wants to merge 86 commits into
motis-project:masterfrom
Jonas-July:timetable-tree-cache

Conversation

@Jonas-July

Copy link
Copy Markdown

Motivation

Currently, loading a timetable means reloading every timetable, which can be very time-consuming.

Using caches and an efficient tree cache structure, this can be avoided, improving loading times considerably at the cost of an initial penalty during cache creation. Subsequent updates can make use of the cached timetables to save on recomputations.

Features

In pursuit of this goal, it is necessary to make the timetable merging explicit (that's currently inplace). This is done granularly to ensure correctness and make debugging easier. These commits consequently make up a large portion of the commits in this PR.
Another feature is the dynamic matching of timetables, whereby the given sources are not just mapped to caches based on their position, but their cache location is chosen depending on the current cache state. This is particularly useful when rearranging sources as is done e.g. when adding and/or removing sources.
The loading tree is implemented as a complete tree (stored in breadth-last order). This is an easy data structure with a logarithmic number of merges when updating.

Performance

TODO
Full transitous data set took 45 min to init the tree, and 8 min on a subsequent update, on my system
The cache takes up 305GiB (roughly 11.3x final size) on disk

Possible improvements

The nodes are still computed sequentially. The tree computation itself can be parallelised to save on time during initial construction; later updates have less impact. There, the merge function can also be parallelised to speed up.
Reading and writing the caches from disk also takes up some time. This can be improved with faster disks, and a fast merge function could allow foregoing the cache (almost) entirely, eliminating the disk writes.

Further information

Further information on the rationale and implementation for the interested reader can be found in the commit messages.
Important milestones are:

  • cfe7583 (Implement change detection)
  • 6402cb3 (Description of merge logic),
  • b2bbeb9 (Enable loading of arbitrary loading trees),
  • e48dba0 (Use efficient complete tree structure), and
  • fa38edf (Dynamic matching of timetables to the cache)

To efficiently store bitfields, they are used and reused across
timetables. A hash_map maintains this mapping to allow quick checking
of existing bitfields.
After loading, this hash_map isn't needed anymore, so the timetable
only stores the mapping from indices to bitfields (as a vector).
From the timetable, it is possible to reconstruct the hash_map. This
eliminates another global variable.

For now, only assert that the reconstruction and the original are
equal.
Since both hash_maps are equal, use the reconstructed one to
eliminate the global variable.
Intermediate timetables can later be reused for caching purposes.
Thus save them inside a cache directory. The naming is chosen such
that each timetable gets its own directory; this may be expanded
in the future.
Useful for hot restart/caching
The change detector is supposed to identify sources that have changed
or rather have not changed since the last run. To be able to make
that determination, it checks that the filepath and config are the
same, and the file hasn't been written to since the last run. This
then implies that the cache file is still valid.
As such, save that information for later use. If saving fails, emit
an error in the log, but don't abort the program since it still
functions, just without the cache.

The timestamp returned by fs::last_write_time is relative to some
filesystem clock which may or may not change at any time. In order
to reliably compare those timestamps, they are converted to the
system time which represents time as the Unix epoch (since C++20,
previously not defined but very likely).
To then store the value, it is converted into a 64bit int representing
the nanoseconds since the Unix epoch.

The configs are hashed and then compared. Hash collisions are negligible.
Since the intermediate data necessary for caching is stored, we
can use it to load from a previous checkpoint when restarting an import.

To find out which sources we can skip, we load the previous change
detector and compare it with the current one. Due to the linear
dependency of importing, we can just check that each source in the list
is unchanged in the order they are presented.
The source arrays must be the same size, otherwise the caches may
overwrite each other, making them invalid.
If a changed source is found (path and/or last write time don't match),
all successive sources will also need recomputing, thus break.

In order to restart from cache, we need to load the cached result.
Due to linearity, again, we can just load the result of the previous
source which hasn't changed and thus would produce equivalent
timetable and shape data.
There could still be issues though. If the cached timetable cannot be
loaded for some reason, we can't use it and have to recompute it
anyway. The same is true if the date_range doesn't match. In that case,
the then previous one has to be loaded (and the one before that, etc.).
If the very first source has to be recomputed, there is no cache left,
and the initial timetable with the special stations is created and
provided.

The timetable is always finalised at the end, so changing finalize_opt
don't have to be considered. If the assistance_times change, the cache
may be invalid and has to be deleted. The code doesn't check for that.

It's probably possible to reorder the input sources to potentially get
more in common with the cached state, but that would make equivalence
a bit harder to prove. The order usually doesn't change anyway or at
least it can be prevented on the caller side.
The shape_idx and shape_offset_idx are relative to the shape storage,
so adjust them when merging.
If the indices are invalid, simply adding them together leads to an
overflow which then produces a valid index. The value however is
wrong and can lead to crashes or silent issues.
If a value is invalid, the adjusted value is also invalid.
The bitfield indices directly impact the timetable in various ways. This
means that the resulting indices need to be adjusted and the references
corrected.

To that end, affected vectors of the original timetable are saved, before
the bitfields_ member is cleared. We can then load the file with an empty
hashmap.

When the file has been loaded, the new traffic_days are calculated. This
is necessary as the old ones use the correct indices, the new ones however
don't and thus need to be adjusted.
Later on, this won't be necessary as such when an empty timetable is passed
to the loader.

Similarly, the old data is restored where relevant to better show the merge.

Finally, merge the new bitfields into the existing timetable and store the
adjusted indices as they might be non-linear. This mapping is then used
to correct the bitfield indices for the traffic days.

No behaviour should change due to this commit.
The location data is grouped by type, thus each vector needs to be
merged separately. A few variables don't get used during loading,
only during finalising, and thus don't need merging (they'll always
be empty).

The routes contain sequences of stops which reference the location_idx.
The stops however only use 28 of the 32 bits which the location_idx_t has,
in order to bitpack the other 4 boolean properties into a single 32 bit type.
This means that the packed value needs to be turned into a stop to then map
the location_idx().

The location index of stops isn't just the location_ variable, because the
location could be invalid. Since the type is shorter, simply comparing against
location_idx_t::invalid() doesn't work. Instead, create a stop from an invalid
value and check if the location_ is the same. Then return location_idx_t::invalid()
to satisfy the interface. Otherwise just recast the location_.

Correspondingly adjusted the indices.
bitvec doesn't have an "append"-like operator, so instead loop over
indices for merging.
route_bikes_allowed_per_section_ and route_cars_allowed_per_section_
are manually resized during loading to correspond in size to
route_idx_t. However, if no routes are added, that resizing doesn't
occur and hence, the vectors might be empty. So prepare them to the
correct size after clearing. The empty values don't get used.
The merge logic doesn't have much to do with loading timetables in
general and is heavily dependent on the structure of timetables.
Thus, move the merge logic into a function inside the timetable class.

There is no change in behaviour expected. For completeness, a description
of the structure of the new method is provided:

1. Some fields don't get used during loading and always remain empty. They
don't need to be merged and aren't handled. This precondition is checked in
case this assumption changes in the future.
2. date_range must be the same between the timetables. It is set from the
outside and it is assumed that at least one timetable has the correct
date_range (and then all of them have if no errors occur).
3. This isn't strictly necessary, but does provide a quick overview of
which fields are handled. They are all read-only of course. The order is
based on the order of definition in timetable.h
4. bitfields and string_idx map have to be merged first to determine the
new indices. They are then passed to the index mapping.
5. The index mapping is a read-only structure which determines the mapping
from the (relative) indices in other_tt to the new absolute indexing.
6. The other fields get merged independently from each other. The mapping
indices only depend on some offsets that can be statically determined
beforehand.

The merging is done structurally, with some exceptions. Each field gets merged
according to its type, until objects are reached which don't need merging as
they stand alone. However, they might contain or be references which need
redirecting to new targets since the merge invalidates most references within
other_tt. This too can be done mostly structurally by mapping the members of
composite types and mapping affected reference types. Care must be taking with
possibly invalid references as they must remain invalid after mapping.
Small refactoring towards a tree like structure
To avoid issues with imports, use explicit namespace notation and
explicitly import the relevant header files.
This becomes useful when only wanting to load changed sources
The new struct `loading_source_node` takes on the responsibility of
loading the timetable from source and returning it.
The node contains all needed information to load the source file
if necessary. The loading, however, can be postponed, enabling
possibly lazy evaluation.
The `loading_tree` structure manages the binary cache tree. This
design allows arbitrary binary trees to be loaded from cache. The
loading is done via recursion over both children.

The nodes are referenced via indices and their relationships are
described by the parents_, left_, and right_ vectors.
The first node (idx=0) is always the default timetable. After that
follow the other source based nodes, and then the intermediate merging
nodes. In total, there are always 2(n+1)-1 nodes, where n is the number
of timetable sources.

Each node has a corresponding cache location to which it will save its
result. By default, it is the node index, but that can be changed if
necessary. To avoid issues between the automatic and the manual indices,
the manual indices can't be set to an automatic value.

Building up the tree has 5 steps:
 - Insert the default node at index 0
 - Insert source nodes up to n+1
 - Determine the updated sources by comparing with the previous cache
 - Insert intermediate nodes. They are updated if either child is updated
 - Write the cache structure to disk. From now on, it is constant.

Loading from the tree-like cache is done via tree traversal. There are
3 types of nodes:
 - The default timetable node at index 0,
 - the loading nodes up to l.source_paths_.size(),
 - the merging nodes after that.

1. The default timetable gets constructed on the fly.
2. The latter two will check whether they need to be recomputed. If they
   don't, they'll try to read the timetable from the cache.
3. If the node needs to be recomputed (either due to update or cache
   failure), we need to determine which kind of node it is:
3.1. Merging nodes call both children recursively. If that fails and they
     return nothing, then they get skipped. If both have been loaded
     successfully, their timetables are merged and written. The shapes
     only get saved if both are available; Usually either both are or both
     are not, as that is dependent on the global setting of whether to
     save the shapes or not. If they get saved, the left gets added first,
     then the right one.
3.2. If it is a source based node, then it will try to load from source.
     On success, it writes the resulting timetable into the cache
     location; The shapes are automatically written to there during the
     loading from source.
     In case the loading fails, this is propagated upwards, where it
     can be skipped.

Finally, the timetable gets finalized. This is unchanged.
Some nodes are source based and thus need access to their underlying
source. Previously, this was handled by fixing the positions relative
to each other. Since the nodes may need to be reordered to facilitate
the tree creation, storing the mapping explicitly makes that easier.

The addition of source based nodes is also now handled inside the
loading_tree, only requiring a simple function call to add a source.
The current tree structure is close to a degenerate binary tree.
Using a complete binary tree is usually more efficient for evenly
spread out updates and is easily computable.

The initial slots are filled with the source based nodes. After
that, the merging nodes are created in a similar manner to binary
heaps. The most obvious difference is the reversed
ordering due to insertion order (starting at the leaves instead of
the root). This algorithm produces a complete binary tree.

The algorithm chooses the two left-most nodes without a parent to
set as children for the next node. This is done until the next node
would be only node left.

This means that the merge order is not necessarily the same as the
ordering inside the array if the source nodes are at different depths
(always the case unless the number of leafs is a power of 2). However,
the default timetable node is required to be the left-most merged
node due to the special stations.
With this tree creation algorithm, it suffices to rotate the sources
by the difference to the next power of 2 (that is the number of leafs
missing on the last level).
Currently, the cache strategy assumes that the cached tree structure
is identical to the current loading tree. Any deviation leads to
cache invalidation and wrong results.
This puts severe restrictions not only on the cache structures that
can be accepted, but also on the number of nodes contained within
them.

In particular, adding or removing sources invalidates the cache
even if the same algorithm is used, leading to extreme performance
degradation.

The dynamic matching of nodes during the tree building checks that
the assumptions are correct and attempts to make good use of the
existing cache. The approach here is not optimal for all existing
tree structures, and doesn't try to adjust the new cache structure
to the existing one. It does, however, produce correct results with
all valid cache tree structures.

---

The matching works on this algorithm:

A merge node can be reused if the underlying sources are unchanged.
The ordering of the sources doesn't need to be checked because any
valid reordering produces essentially the same timetable. Thus,
the sources can be sorted for easier matching. Due to the tree
structure, the sets of children are unique for each node.

Source based nodes are the same if the source data is unchanged.
Thus, the path can be used to quickly determine a match candidate.

The default timetable node has the reserved cache index of 0.

When a match has been found, it's cache index is transferred to the
new node. That index might already be used by another node, in which
case they swap them. Since each cache index is unique, this swap
doesn't mess with the matching.

---

In summary, the loading from source is only done with new sources,
while cached merging nodes may or may not be useful depending on
the differences in structure.
Best performance is achieved when prepending sources on the left.
@felixguendling
felixguendling force-pushed the master branch 2 times, most recently from a4efa76 to 116082c Compare December 11, 2025 14:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant